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FOREWORD 


The  United  States  Army  Reserve  (USAR)  is  looking  for  more  effective  and  efficient 
ways  to  train  and  evaluate  rifle  marksmanship  through  the  use  of  simulation  devices.  To  this  end, 
and  at  the  request  of  the  U.S.  Army  Reserve  Command  (USARC),  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  has  been  working  with  the  USAR’s  84th 
Institutional  Training  Division  (DIVIT)  and  Small  Arms  Readiness  Group  (SARG)  to  develop 
and  evaluate  a  simulation-based  (i.e.,  the  Beamhit™  Laser  Marksmanship  Training  System 
[LMTS])  training  and  evaluation  program  for  use  at  home  station  (i.e.,  reserve  centers).  The 
common  goal  of  this  cooperative  effort  is  to  field  a  program  that  will  produce  marksmanship 
proficiency  levels  that  meet  or  exceed  unit  readiness  requirements  while  minimizing  the 
resources  needed  to  do  so. 

To  date,  a  Program  of  Instruction  (POI)  has  been  developed  to  cover  the  use  of  LMTS  in 
training  rifle  marksmanship  fundamentals  (e.g.,  steady  position,  aiming,  breath  control,  and 
trigger  squeeze).  A  necessary  step  in  the  implementation  of  this  POI  is  to  determine  the  relation 
between  LMTS-  and  live-fire-based  marksmanship  performance  and,  if  found  to  be  of  sufficient 
magnitude,  develop  tools  for  trainers  to  use  in  predicting  the  latter  from  the  former.  This  report 
describes  research  conducted  to  assess  this  relation  and  then  develop  separate  tools  for  predicting 
Army  standard  pop-up  target  qualification  course  scores  from  LMTS  dry  vs.  Blazer-based  modes 
of  fire. 


This  research  was  conducted  by  the  ARI-Reserve  Component  Training  Research  Unit, 
whose  mission  is  to  improve  the  effectiveness  and  efficiency  of  Reserve  Component  (RC) 
training  through  use  of  the  latest  in  training  technology.  This  research  is  supported  under  Work 
Package  219,  "Strategies  and  Tools  for  Maximizing  AC/RC  Unit  Performance,"  of  ARI's  Science 
and  Technology  Program  for  Fiscal  Year  2002. 

This  research  was  sponsored  by  USARC  under  a  continuing  Memorandum  of 
Understanding  initially  signed  12  June  1985.  Findings  have  been  presented  to  the  Weapons 
Program  Administrator,  USARC;  and  Deputy  Chief  of  Staff  for  Operations,  84th  DIVIT,  USAR. 


KATHLEEN  A.  QUINKERT 
Acting  Technical  Director 
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USING  THE  LASER  MARKSMANSHIP  TRAINING  SYSTEM  TO  PREDICT  RIFLE 
MARKSMANSHIP  QUALIFICATION 

EXECUTIVE  SUMMARY _ _ 


Research  Requirement: 

To  develop  Laser  Marksmanship  Training  System  (LMTS)  dry-fire-  and  Blazer-Based 
(i.e.,  sound/recoil  replicator)  tools  for  predicting  first-run  rifle  marksmanship  performance  on 
the  Army’s  standard,  pop-up  target  qualification  course. 

Procedure: 

A  total  of  186  Idaho  and  Oregon  Reserve  Component  (RC)  soldiers  fired  for  qualification 
on  a  scaled  LMTS  version  and  live-fire  version  of  the  Army’s  standard  pop-up  target 
qualification  course.  LMTS  was  fired  under  either  a  dry-fire  mode,  where  shooters  had  to 
manually  cock  their  rifles  after  firing  each  laser  round,  or  a  Blazer  mode,  where  nonpolluting 
blanks  provided  the  recoil  and  noise  associated  with  live  rounds.  Separate  regression  analyses 
were  performed  to  identify  the  relation  between  live-fire  scores  and  those  obtained  under  the 
two  LMTS  firing  modes.  The  identified  relations  were  then  used  to  develop  separate  LMTS- 
based  tools  for  predicting  the  probability  of  soldier  live-fire  qualification. 


Findings: 

Statistically  significant  positive  linear  relations  were  found  (and  then  validated)  between 
first-run  live-fire  scores  and  both  LMTS  dry-fire-  (r  =  .50)  and  Blazer-based  (r  =  .55)  scores. 
These  relations  were  of  sufficient  strength  to  permit  development  of  easy-to-use  tools  for 
predicting  first-run,  live-fire  qualification  with  up  to  90%  confidence  using  the  former  and 
with  up  to  80%  confidence  using  the  latter. 


Use  of  Findings: 

With  these  tools,  RC  marksmanship  trainers  can  implement  a  competency-based  training 
program  where  soldiers  most  in  need  of  remedial  training  (i.e.,  poor  shooters)  can  be  quickly 
identified,  and  the  point  at  which  sufficient  training  has  been  provided  (i.e.,  when  first-run 
live-fire  qualification  is  likely)  easily  determined.  These  tools  also  provide  RC  unit 
commanders  with  empirically  derived  live-fire  performance  standards  needed  to  support  use 
of  LMTS  in  place  of  live-fire  for  rifle  marksmanship  proficiency  validation  purposes  when 
standard  pop-up  target  course  range  facilities  are  not  readily  available.  Although  both  tools 
will  serve  these  purposes,  that  based  on  LMTS  dry-fire  is  recommended  because  of  the  added 
expense  of  firing  with  Blazer  without  an  accompanying  statistically  significant  increased 
predictive  benefit. 
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Using  the  Laser  Marksmanship  Training  System  to  Predict 
Rifle  Marksmanship  Qualification 

Introduction 

Tightening  budgets,  escalating  ammunition  costs,  dwindling  numbers  of  certified  live-fire 
ranges,  and  ever-present  training  time  constraints  have  prompted  the  U.S.  Army  Reserve 
(USAR)  to  search  for  alternative  ways  to  train  and  evaluate  rifle  marksmanship  through  the  use 
of  simulation.  To  this  end,  the  U.S.  Army  Research  Institute  (ARI)  has  been  working  in 
partnership  with  the  U.S.  Army  Reserve  Command’s  marksmanship  executive  agent,  the  84* 
Division  (Institutional  Training),  to  develop  and  evaluate  a  simulation-based  rifle  marksmanship 
sustainment  training  program  designed  for  use  at  home  station.  The  goal  of  this  effort  is  to 
produce  shooter  proficiency  levels  that  meet,  or  beat,  unit  readiness  requirements  while 
minimizing  the  time  and  ammunition  needed  to  do  so  (Plewes,  1997,  November  24). 

The  envisioned  program  of  marksmanship  instruction  (POI)  calls  for  use  of  the  Laser 
Marksmanship  Training  System  (LMTS)  (BeamHit,  1999;  Dunlin,  1999,  January-February)  to 
support  (a)  the  training  of  marksmanship  fundamentals  (ie.,  steady  position,  aiming,  breath 
control,  and  trigger  squeeze),  shot  grouping/weapon  zeroing  practice,  and  simulated  record  fire 
evaluation,  (b)  competency-based  exercise  delivery  where  LMTS-based  pretesting  is  used  to 
predict  which  soldiers  need  training  (Le.,  unlikely  live-fire  qualifiers)  and  posttesting  is  used  to 
signal  when  enough  such  training  has  been  provided  (i.e.,  once  live-fire  qualification  becomes 
likely),  and  (c)  use  of  LMTS-based  testing  to  validate  live-fire  qualification  status  when  range 
facilities  are  not  readily  available. 

The  POI  exercises  devoted  to  training  marksmanship  fundamentals  have  been  completed  and 
published  in  Appendix  C  of  the  coordinating  draft  of  Field  Manual  (FM)  3.22.9  (23-9)  “Rifle 
Marksmanship  M16A1,  M16A2/3,  M16A4,  and  M4  Carbine”  (Headquarters,  Department  of  the 
Army,  2001,  August).  In  addition,  an  LMTS-based,  live-fire  prediction  tool  has  been  developed 
to  provide  unit  commanders  with  the  needed  yardstick  upon  which  to  base  pre-  and  posttesting, 
as  well  as  validate  live-fire  qualification  status.  Current  predictions  only  apply,  however,  to 
scores  fired  on  the  Army’s  25m  Alternate  Qualification  Course  (ALT  C)  where  shooters  fire  at 
stationary  paper  targets  containing  E-  and  F-Type  silhouettes  scaled  to  represent  known  down- 
range  distances  of  from  50-300m  (e.g.,  Headquarters,  Department  of  the  Army,  1989;  Smith  & 
Hagman,  2000). 

Although  most  USAR  soldiers  typically  fire  for  qualification  on  ALT  C  because  of  its  limited 
space  and  instrumentation  requirements,  firing  on  the  Army’s  standard  pop-up  target 
qualification  course,  often  accomplished  using  a  Remote  Electronic  Target  System  (RETS),  is 
preferred  whenever  possible  because  of  its  demand  for  a  greater  array  of  combat-related 
marksmanship  skills  (e.g.,  target  acquisition).  To  produce  a  corresponding  demand  for  these 
skills  during  simulation-based  qualification  firing,  BeamHit  (2002)  has  developed  an  LMTS- 
based,  pop-up  target  qualification  course  called  “mini-RETS.”  Research,  similar  to  that 
completed  with  ALT-C,  is  therefore  needed  to  determine  what  the  relation  is  between  simulated 
and  live-fire  pop-up  target  course  performance  and  whether  predictions  of  the  latter  from  the 
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former  are  accurate  enough  to  support  LMTS-based  pretesting,  posttesting,  and  a  record  fire 
qualification/validation  alternative  to  live  fire. 

Current  live-fire  predictions  are  also  based  on  an  LMTS  dry-fire  mode  of  target  engagement 
where  shooters  manually  cock  their  weapons  before  firing  each  simulated  (i.e.,  laser)  round.  To 
add  realism  to  this  firing  experience,  and  potentially  boost  predictive  accuracy  above  that 
associated  with  dry  fire,  a  recoil  and  sound  replicator  enhancement  to  LMTS,  called  the  Blazer, 
has  been  developed  by  the  device’s  manufacturer.  The  Blazer  provides  a  dry-fire  alternative  that 
captures  fully  simulated  live-fire  functionality,  using  nonpolluting  blanks  in  place  of  live 
ammunition,  with  nearly  100%  of  the  associated  recoil  and  50%  of  the  noise. 

The  question  is  whether  this  added  capability  results  in  better  live-fire  predictive  accuracy 
than  that  achieved  under  dry  fire.  Research  is  needed,  therefore,  to  identify  the  relation  between 
Blazer-  and  live-fire-based  performance,  develop  and  validate  a  Blazer-based  tool  for  predicting 
the  latter  from  the  former  for  the  Army’s  standard  pop-up  target  qualification  course,  and 
compare  the  relative  accuracy  of  Blazer-  vs.  dry-fire-based  predictions.  The  present  research 
provides  this  information. 


Experiment  1 

This  experiment  examined  the  relation  between  LMTS-based,  dry-fire  scores  fired  on  mini- 
RETS  and  live-fire  scores  fired  on  an  Army-certified,  pop-up  target  qualification  course. 
Experiment  2  examined  the  relation  between  Blazer-based  and  live-fire  scores. 

Method 

Participants 

One-hundred  and  ten  soldiers  from  an  Idaho  Army  National  Guard  armored  cavalry  brigade 
support  battalion  voluntarily  participated  in  the  research  as  part  of  yearly  rifle  qualification  firing 
at  Orchard  Range  near  Boise,  Idaho.  None  of  the  participants  had  prior  experience  firing  LMTS. 

Equipment 

LMTS  is  a  laser-emitting  device,  designed  primarily  for  indoor  use,  which  enables  soldiers  to 
engage  targets  without  the  use  of  live  ammunition.  Its  major  components  include  a  laser 
transmitter,  a  mandrel  to  which  the  transmitter  is  attached/aligned,  a  variety  of  laser-sensitive 
targets,  and  a  dedicated  computer  with  optional  printer  (Figure  1).  One  end  of  the  mandrel  holds 
the  laser  transmitter  while  the  other  end  slips  into  the  barrel  of  the  weapon.  Vibrations  from  the 
weapon’s  mechanical  firing  mechanism  activate  the  laser  when  the  trigger  is  pulled  and  the 
location  of  the  emitted  beam  is  “picked  up”  by  the  laser-sensitive  target(s)  and  recorded/stored 
on  the  computer  for  future  analysis  and  printout.  Soldiers  can  shoot  with  LMTS  under  a  dry-fire 
mode  using  their  own  weapons,  or  they  can  replace  the  upper  receiver  assembly  with  the  Blazer 
(Figure  2)  to  enable  weapon  bolt  and  ejector  mechanisms  to  function  with  nontoxic  blank 
ammunition.  In  this  first  experiment,  soldiers  dry  fired  their  own  M16A2  weapons  on  the  scaled 
1 5m  LMTS  mini-RETS  pop-up  target  qualification  course  shown  in  Figure  3 . 
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Figure  1.  LMTS  computer/monitor,  sample  electronic  target,  and  laser  transmitter  with  attached 
mandrel. 


Figure  2.  LMTS  Blazer. 


Figure  3.  Soldier  firing  on  the  15m  LMTS  mini-RETS  range. 

Procedure 

To  control  for  possible  sequence  affects,  approximately  half  the  soldiers  fired  for  live-fire 
qualification  first  and  LMTS-based  qualification  second.  The  other  half  fired  this  sequence  in 
reverse,  with  no  more  than  24  hr  occurring  between  the  two  firings  under  either  sequence. 

Live-Fire  qualification  Dedicated  cadre  from  the  Idaho  Army  National  Guard  conducted  all 
range-based  qualification  firing  in  accordance  with  procedures  stipulated  in  FM  23-9 
(Headquarters,  Department  of  the  Army,  1989)  with  shot  grouping  and  weapon  zeroing 
accomplished  immediately  beforehand.  All  pop-up  targets  were  automatically  presented  and 
scored.  Hit  numbers  associated  with  specific  shooting  classifications  were  as  follows:  0-22, 
Unqualified;  23-29,  Marksman;  30-35,  Sharpshooter;  36-40  Expert.  All  statistical  analyses  were 
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based  on  first-run  hit  scores  (Le.,  the  number  of  targets  hit,  out  of  a  possible  40,  on  the  soldiers’ 
first  qualification  attempt). 

LMTS-Based  qualification.  Simulated  qualification  firing  with  LMTS  was  conducted 
indoors  by  dedicated  cadre  from  the  USAR’s  84th  Division  and  Small  Arms  Readiness  Group 
who  followed  the  same  procedures  as  those  associated  with  live  fire  (Headquarters,  Department 
of  the  Army,  1989).  LMTS  was  fired  in  a  dry-fire  mode  that  required  soldiers  to  use  their  firing 
hand  to  manually  cock  their  weapons  after  each  round  by  recycling  the  charging  handle  located 
at  the  rear  of  the  upper  receiver  assembly.  All  LMTS  targets  were  presented  and  scored 
automatically. 

Analytic  Approach 

To  determine  the  relation  between  LMTS-  and  live-fire-based  qualification  scores,  a  split- 
group,  cross-validation  design  was  used  (Tatsuoka,  1969).  The  initial  sample  of  1 10  soldiers  was 
randomly  divided  into  two  groups.  Group  1  was  used  to  develop  a  prediction  equation  between 
device  and  live-fire  scores.  This  equation  was  then  applied  to  Group  2  soldiers  to  see  if  their  live- 
fire  scores  could  be  predicted  successfully  from  the  equation  derived  from  Group  1  data. 

Results 


Group  1  Data  (n  =  55) 

For  Group  1,  LMTS  hit  scores  ranged  from  7  to  39  (M=  27.8,  SD  =  7.7).  Live-Fire  hit  scores 
ranged  from  10  to  37  (M=  27.5,  SD  =  5.8).  A  paired-samples  Mest  revealed  that  the  mean  hit 
scores  for  the  two  firing  methods  did  not  differ  significantly.  The  rejection  region  for  this  and  all 
other  statistical  analyses  was  .05. 

A  least-squares  regression  prediction  equation  of  the  form  Y'  =  Bo  +  Bi(Xi)  was  developed 
in  which  Y'  was  the  predicted  live-fire  criterion  score.  Bo  was  the  intercept  (or  theoretical  live- 
fire  score  when  the  LMTS  score  equals  zero),  Bi  was  the  empirically  derived  regression 
coefficient  linking  changes  in  the  live-fire  criterion  variable  with  changes  in  the  LMTS  predictor 
variable,  and  Xi  was  the  obtained  LMTS  first-run  qualification  score.  A  significant  linear 
relation,  Y'  =  16.485  +  .395(Xi),  SE  =  2.55  was  found  between  LMTS  and  live-fire 
performance,  F(l,  53)  =  19.96.  In  addition,  the  correlation  between  predicted  and  actual  live-fire 
scores  was  significant  (r  =  .52).  Thus,  Group  1  LMTS  scores  were  both  linearly  related  to,  and 
reasonably  good  predictors  of  live-fire  scores. 

Group  2  Data  (n  =  55) 

For  Group  2,  LMTS  scores  ranged  from  3  to  39  (M=  28.4,  SD  —  7.5).  Live-fire  scores  ranged 
from  8  to  39  (M  =  26.9,  SD  =  5.9).  As  with  Group  1,  Group  2  mean  LMTS  and  live-fire  scores 
did  not  differ  significantly. 

Following  cross-validation  procedures  described  by  Tatsuoka  (1969),  the  Group  1  regression 
equation  was  used  to  predict  Group  2  live-fire  scores,  and  then  the  relative  amount  of  variance 
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accounted  for  in  each  group  was  compared.  A  significant  linear  relation,  Y'  =  .145  +  .966(Xi 
[predicted]),  SE  =  6.56,  was  found  between  actual  live-fire  scores  and  live-fire  scores  predicted 
from  the  equation  based  on  Group  1  data,  F( 1,  53)  =  16.33.  The  resulting  correlation  (r  =  .49) 
was  significant,  and  the  associated  Group  2 1?2  of  .236  did  not  differ  significantly  (z  —  .267)  from 
the  R2  of  .274  found  for  Group  1,  indicating  that  the  Group  1  prediction  equation  accounted  for  a 
comparable  amount  of  live-fire  score  variance  in  the  two  groups.  Thus,  the  predictive  model  was 
found  to  be  valid  and,  therefore,  likely  to  maintain  similar  efficiency  when  used  to  predict  live- 
fire  scores  of  other  soldier  samples. 

Pooled  Data  (N  =  110) 

The  significant  linear  relation  found  among  Group  1  soldiers  was  successfully  cross- 
validated  on  Group  2.  Given  the  similar  outcomes  from  these  two  analyses,  and  the 
generalizability  of  the  preliminary  model  based  on  only  half  the  sample,  data  from  the  two 
groups  were  pooled  in  order  to  develop  the  best  possible  live-fire  predictions. 

Order  effects.  Approximately  half  the  soldiers  fired  LMTS  first  and  then  live  fired.  For  the 
other  soldiers,  this  sequence  was  reversed.  Both  LMTS  and  live-fire  scores  were  examined  to 
determine  if  they  had  been  affected  by  firing  order.  In  neither  case  was  there  a  significant  effect. 
Therefore,  no  fUrther  mention  of  firing  order  will  be  made. 

Descriptives  and  mean  differences.  LMTS  scores  ranged  from  3  to  39  ( M  -  28.1,  SD  —  7.5). 
The  minimum  qualification  score  of  23  was  achieved  on  LMTS  by  80.9%  of  soldiers  on  their 
first  qualification  attempt  (Ql).  Live-fire  scores  ranged  from  8  to  39  (M=  27.2,  SD  =  5.8)  with 
ninety-five  of  the  soldiers  (86.4%)  achieving  live-fire  Ql.  A  paired-samples  r-test  revealed  that 
live-fire  and  LMTS  mean  scores  did  not  differ  significantly. 

Prediction  model.  A  least-squares  regression  analysis  revealed  a  significant  linear  relation, 
F(l,  108)  =  36.29,  between  the  predictor  variable  (LMTS  score)  and  the  criterion  variable  (live- 
fire  score).  The  correlation  (r  =  .50)  between  LMTS  and  live-fire  fire  scores  was  significant,  with 
the  former  accounting  for  alx>ut  a  fourth  of  the  variance  in  the  latter  (R2  =  .25;  adjusted  j?2  =  .24). 
The  prediction  equation  took  the  form  Y'  =  16.323  +  .387(Xi),  SE  —  1 .87,  where  Y'  =  predicted 
live-fire  score,  Xi  =  LMTS  score,  and  SE  =  standard  error  of  the  prediction. 

For  this  obtained  relation  to  be  of  practical  value,  it  should  allow  marksmanship  trainers  to 
predict  live-fire  qualification  scores/probabilities  based  on  LMTS  dry-fire  performance. 
Specifically,  trainers  need  to  know  what  levels  of  LMTS  performance  are  associated  with  live- 
fire  cut  scores  of  23  (Marksman),  30  (Sharpshooter)  and  36  (Expert).  To  determine  the  LMTS 
score  associated  with  these  scores,  we  took  the  pooled  data  prediction  equation, 

Y'  =  16.323  +  .387(Xi) 

substituted  the  desired  live-fire  cut  score  for  Y',  and  solved  for  the  required  score  on  LMTS. 
Thus,  to  determine  the  LMTS  score  associated  with  a  live-fire  score  of  23,  we  found: 

23  =  16.323  +  .387(Xi) 


5 


6.68  =  .387(Xi) 
(X0  =  17.25 


Soldiers  with  an  LMTS  score  of  17  will,  on  average,  obtain  a  live-fire  score  of  23,  the 
minimum  score  required  for  qualification  at  the  Marksman  level.  Not  all  soldiers  with  an  LMTS 
score  of  17  will  obtain  a  live-fire  score  of  exactly  23,  of  course.  Some  will  score  lower  than  23 
and  some  will  score  higher,  but  their  average  score  will  be  23.  Similarly,  it  can  be  determined 
on  average,  an  LMTS  score  of  35  will  be  associated  with  a  live-fire  score  of  30,  the 
minimum  score  needed  for  Sharpshooter  qualification. 

30  =  16.323  +  .387(Xi) 

13.677  =  .387(X0 
(Xi)  =  35.34 

Unfortunately,  insufficient  data  in  the  extreme  regions  of  the  two  scales  were  available  to 
permit  the  accurate  prediction  of  Expert  qualification.  The  required  LMTS  score  lies  outside  the 
range  of  possible  scores  (i.e.,  0  to  40)  and  indicates  that  soldiers  wishing  to  Q1  as  experts  had 
best  fire  the  highest  score  possible  on  LMTS. 

36  =  16.323  +  .387(Xi) 

1 9.677  =  .387(X0 
(Xi)  =  50.84 

Assuming  that  the  actual  probability  of  firing  these  predicted  live-fire  scores  will  follow  a 
normal  distribution  with  approximately  equal  variability  (Hays,  1963,  p.  523.)  and  M=  23, 30, 
and  36,  the  probability  of  an  individual  soldier  shooting  each  of  these  live-fire  scores  or  higher 
was  calculated  using  the  ARI  Live-Fire  Prediction  Tool  (Hagman,  1998).  Table  1  shows  the 
results.  A  soldier  with  an  LMTS  score  of  24  (Column  1),  for  instance,  would  be  predicted  to  fire 
26  on  the  live-fire  range  (Column  2)  and  have  a  70%  chance  of  successful  record  fire 
qualification  at  the  Marksman  level  (Column  3),  a  20%  chance  of  qualification  at  the 
Sharpshooter  level  (Column  4),  and  less  than  a  10%  chance  of  qualification  at  the  Expert  level 
(Column  5).  A  soldier  with  an  LMTS  score  of  28  would  be  predicted  to  fire  27  on  the  range  and 
have  an  80%  chance  of  qualifying  Marksman,  a  20-30%  chance  of  qualifying  Sharpshooter,  and 
less  than  a  10%  chance  of  qualifying  as  an  Expert,  and  so  forth. 

Another  way  of  using  Table  1  is  to  read  down  one  of  the  last  three  columns  to  the  desired 
)  level  of  live-fire  qualification  probability,  and  then  across  to  the  first  column  to  determine  the 
LMTS  score  associated  with  that  probability.  For  example,  if  a  marksmanship  trainer  wants  a 
soldier  to  have  at  least  a  50%  chance  of  record  fire  qualification,  then  an  LMTS  score  of  17  is 
required.  The  chance  of  record  fire  qualification  rises  to  70%  if  the  soldier’s  LMTS  score  is  24, 
and  it  climbs  to  90%  with  an  LMTS  score  of  34. 

In  summary,  the  observed  predictive  correlation  (r  =  .50)  between  LMTS-  and  live-fire-based 
scores  was  both  reliable  and  generalizable,  as  indicated  by  the  successful  cross-validation 
procedure  using  randomly  constituted  subgroups  of  the  total  sample.  The  relation  was  also  of 
sufficient  strength  to  be  of  practical  value  in  predicting  live-fire  qualification  performance. 
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Table  1. 

LMTS-Based  Predicted  Chances  of  First-Run  Qualification  at  Marksman  (223  hits). 
Sharpshooter  (2  30  hits),  and  Expert  (236  hits)  Levels  on  a  Standard  Pop-Up  Target 
Qualification  Course _ 


Chances  of  a  Live-Fire  Score  of , . . 


LMTS 

Predicted  Mean 

2. 23 

>  30 

>  36 

Score 

Record  Fire  Score 

Marksman 

Sharpshooter 

Expert 

16 

10 

— 

— 

6 

19 

20 

~ 

— 

10 

20 

30 

— 

14 

22 

40 

— 

— 

17 

•  23 

50 

— 

— 

19 

23 

— 

10 

— 

21 

24 

60 

— 

24 

26 

70 

20 

— 

28 

27 

80 

— 

— 

29 

27 

— 

30 

— 

32 

29 

— 

40 

— 

34 

29 

90 

~ 

10 

35 

30 

— 

50 

— 

39 

31 

— 

60 

~ 

40 

32 

— 

— 

20 

Experiment  2 

This  experiment  examined  the  relation  between  LMTS  Blazer-Based  mini-RETS  scores  and 
live-fire  scores  collected  on  a  standard  Army  pop-up  qualification  course  employing  RETS 
technology. 

Method 


Participants 

Seventy-six  soldiers  from  an  Oregon  Army  National  Guard  infantry  battalion  voluntarily 
participated  in  the  research  as  part  of  yearly  rifle  qualification  firing  at  Camp  Rilea,  Oregon. 
None  of  the  participants  had  previous  experience  firing  LMTS. 

Procedure 

To  control  for  possible  order  effects,  approximately  half  the  soldiers  fired  for  live-fire 
qualification  first  and  LMTS-based  qualification  second.  The  other  half  fired  this  sequence  in 
reverse,  with  no  more  than  24  hr  occurring  between  the  two  firings  under  either  sequence. 

Live-Fire  qualification.  Live-Fire  qualification  was  fired  with  the  M4  Carbine  (using  backup 
iron  sights)  and  conducted  by  dedicated  range  cadre  from  the  Oregon  Army  National  Guard  in 
accordance  with  procedures  stipulated  in  FM  23-9  (Headquarters,  Department  of  the  Army, 
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1989)  with  shot  grouping  and  weapon  zeroing  accomplished  immediately  beforehand.  Target 
presentation  and  scoring  were  automated.  Hit  numbers  associated  with  specific  shooting 
classifications  were  as  follows:  0-22,  Unqualified;  23-29,  Marksman;  30-35,  Sharpshooter;  36- 
40,  Expert.  All  statistical  analyses  were  based  on  first-run  hit  scores. 

LMTS-Based  qualification.  Simulated  qualification  firing  with  the  LMTS  M4  Carbine  Blazer 
(Figure  4)  was  conducted  on  a  scaled,  indoor,  15m  mini-RETS  range  by  dedicated  cadre  from 
the  USAR’s  84th  and  98th  Institutional  Training  Divisions.  Blazer  firing  followed  the  same 
procedures  as  those  used  for  live  fire  and  employed  automated  target  presentation  and  scoring. 

Analytic  Approach 

A  split-group,  cross-validation  design  was  used  to  examine  the  relation  between  LMTS 
Blazer-  and  live-fire  performance.  The  initial  sample  of  76  soldiers  was  randomly  divided  into 
two  groups.  Group  1  was  used  to  develop  a  prediction  equation  between  Blazer-  and  live-fire 
scores.  This  equation  was  then  applied  to  Group  2  soldiers  to  see  if  their  live-fire  scores  could  be 
predicted  successfully  from  an  equation  based  on  Group  1  data 


Figure  4.  Soldiers  firing  with  the  M4  Carbine  Blazer. 


Results 


Group  1  Data  (n=  38) 

For  Group  1,  Blazer  hit  scores  ranged  from  8  to  39  (M=  28.8,  SD  =  8.4).  Live-fire  scores 
ranged  from  2  to  36  (M=  24. 1,  SD  =  7.0).  A  paired-samples  /-test  revealed  that  the  mean  number 
of  Blazer  hits  was  significantly  greater  than  that  achieved  with  live-fire,  t{ 37)  =  4.00. 

A  least-squares  regression  prediction  equation  of  the  form  Y'  =  Bo  +  Bi(Xi)  was  developed 
in  which  Y'  was  the  predicted  live-fire  criterion  score.  Bo  was  the  intercept,  Bi  was  the 
empirically  derived  regression  coefficient  linking  changes  in  the  live-fire  criterion  variable  with 
changes  in  the  Blazer  predictor  variable,  and  Xi  was  the  obtained  Blazer  hit  score.  A  significant 
linear  relation,  Y'  =  10.675  +  .466(X0,  SE  =  3.42  was  found  between  Blazer  and  live-fire 
performance,  F(l,  36)  =  16.74.  In  addition,  the  correlation  (r  =  .56)  between  predicted  and  actual 
live-fire  scores  was  significant.  Thus,  Group  1  Blazer  scores  were  both  linearly  related  to,  and 
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reasonably  good  predictors  of,  live-fire  performance  despite  the  finding  that  soldiers  fired  better 
with  Blazer  than  with  live  ammunition. 

Group  2  Data  (n  =  28) 

For  Group  2,  Blazer  hit  scores  ranged  from  13  to  40  (M=  31.1,  SD  =  6.3).  Live-fire  scores 
ranged  from  9  to  35  (M  =  24.4,  SD  =  5.6).  As  found  for  Group  1,  the  mean  Blazer  score  was 
significantly  greater  than  the  mean  live-fire  score,  /( 37)  =  7.04.  Thus,  Blazer  scores  were 
significantly  higher  than  live-fire  scores  in  both  groups. 

Following  the  same  cross-validation  procedures  used  earlier,  the  Group  1  regression  equation 
was  used  to  predict  Group  2  live-fire  scores,  and  then  the  relative  amount  of  variance  accounted 
for  in  each  group  was  compared.  A  significant  linear  relation,  Y'  =  -1.021  +  1.0(Xi  [predicted]), 
SE  =  6.56,  was  found  between  actual  live-fire  scores  and  live-fire  scores  predicted  (Xi)  from  the 
equation  based  upon  Group  1  data,  F(l,  36)  =  13.85.  The  resulting  correlation  (r  =  .53)  was 
significant,  and  the  associated  Group  2  if2  of  .278  did  not  differ  significantly  (z<  1)  from  the  X2 
of  .3 1 7  found  for  Group  1 ,  indicating  that  the  Group  1  prediction  equation  accounted  for  a 
comparable  amount  of  live-fire  score  variance  in  the  two  groups.  Thus,  the  predictive  model  was 
found  to  be  valid,  in  spite  of  mean  differences  between  Blazer  and  live-fire  scores  and,  therefore, 
likely  to  maintain  similar  efficiency  when  used  to  predict  live-fire  scores  of  other  soldier 
samples. 

Pooled  Data  (N  =  76) 

The  significant  linear  relation  found  among  Group  1  soldiers  was  successfully  cross- 
validated  on  Group  2.  Given  the  similar  outcomes  from  these  two  analyses,  and  die  apparent 
generalizability  of  the  preliminary  model  based  on  only  half  the  sample,  data  from  the  two 
groups  were  pooled  in  order  to  develop  the  best  possible  live-fire  predictions. 

Order  effects.  Approximately  half  the  soldiers  fired  Blazer  first  and  then  live  fired.  For  the 
other  soldiers,  this  sequence  was  reversed.  Both  Blazer  and  live-fire  scores  were  examined  to 
determine  if  they  had  been  influenced  by  firing  order.  In  neither  case  was  there  a  significant 
effect.  Therefore,  no  further  mention  of  firing  order  will  be  made. 

Descriptives  and  mean  differences.  Blazer  scores  ranged  from  8  to  40  (M-  29.9,  SD  —  7.5). 
The  minimum  Ql  qualification  score  of  23  was  achieved  with  Blazer  by  85.5%  of  the  soldiers  in 
the  pooled  sample.  Live-fire  scores  ranged  from  2  to  36  (M  =  24.2,  SD  =  6.3).  Fifty-two  soldiers 
(68.4%)  achieved  live-fire  Ql .  A  paired-samples  /-test  revealed  that  the  mean  hit  scores  found 
for  Blazer  were  reliably  higher  than  those  found  for  live-fire,  /(75)  =  7.50. 

Prediction  model.  A  least-squares  regression  analysis  revealed  a  significant  linear  relation, 
F(l,  74)  =  31.46,  between  the  predictor  variable  (Blazer  hit  score)  and  the  criterion  variable 
(live-fire  hit  score).  The  correlation  (r  —  .55)  between  Blazer  and  live-fire  scores  was  significant, 
with  the  former  accounting  for  more  than  a  fourth  of  the  variance  in  the  latter  (R2  =  .30;  adjusted 
R2  =  .29).  The  prediction  equation  took  the  form  Y'  =  10.470  +  .460(Xi),  SE  =  2.53,  where  Y" 

=  predicted  live-fire  score,  Xi  =  Blazer  score,  and  SE  —  standard  error  of  the  prediction. 
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For  this  relation  to  be  of  practical  value,  it  should  allow  marksmanship  trainers  to  predict 
live-fire  qualification  scores/probabilities  based  on  Blazer  performance.  Specifically,  trainers 
need  to  know  what  levels  of  Blazer  performance  are  associated  with  live-fire  qualification  cut 
scores  of  23  (Marksman),  30  (Sharpshooter),  and  36  (Expert).  To  determine  the  Blazer  score 
associated  with  these  live-fire  scores,  we  took  the  pooled  data  prediction  equation, 

Y'  =10.470  +  .460(X0 

substituted  the  desired  live-fire  cut  score  for  Y',  and  solved  for  the  required  Blazer  score.  Thus, 
to  determine  the  Blazer  score  associated  with  a  live-fire  score  of  23,  we  found: 

23  =  10.470 +  .460(Xi) 

12.53  =  .460(X,) 

(Xi)  =  27.24 

Soldiers  with  a  Blazer  score  of  27  will,  on  average,  obtain  a  live-fire  score  of  23,  the 
minimum  required  for  qualification  at  the  Marksman  level.  Of  course,  not  all  soldiers  with  a 
Blazer  score  of  27  will  obtain  a  live-fire  score  of  exactly  23.  Some  will  score  lower  than  23  and 
others  will  score  higher,  but  their  average  score  will  be  23. 

Unfortunately,  further  examination  revealed  that  the  model  cannot  successfully  predict  live- 
fire  performance  levels  above  Marksman.  For  example,  when  we  plugged  in  a  live-fire  score  of 
30,  the  minimum  live-fire  score  needed  for  Sharpshooter  qualification,  we  obtained  a  Blazer 
score  of  42,  which  lies  outside  the  range  of  possibility. 

30=  10.470  +  .460(X0 

19.53  =  .460(X0 
(Xi)  =  42.46 

An  even  more  extreme  outcome  occurred  when  we  calculated  the  Blazer  score  necessary  to 
predict  live-fire  scores  of  36,  the  minimum  necessary  score  for  Expert  qualification 

36  =  10.470  +  .460(Xi) 

25.53  =  .460(X0 
(Xi)  =  55.50 

The  resulting  score  of  55  was  well  outside  the  possible  range  of  Blazer  scores.  Thus,  soldiers 
interested  in  live-fire  qualifying  as  either  Sharpshooter  or  Expert  would  be  advised  to  train  to  the 
highest  possible  LMTS  Blazer  score. 

ARI  Prediction  Tool  results  (Table  2)  for  LMTS  Blazer  data  reveal  that  a  soldiers  with  an 
LMTS  Blazer  score  of  33  (Column  1),  for  instance,  would  be  predicted  to  fire  26  on  the  live-fire 
range  (Column  2)  and  have  a  70%  chance  of  successful  qualification  at  the  Marksman  level 
(Column  3),  a  20%  to  30%  chance  of  qualification  at  the  Sharpshooter  level  (Column  4),  and  less 
than  a  10%  chance  of  qualification  at  the  Expert  level  (Column  5).  From  the  last  two  columns  in 
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Table  2,  it  can  also  be  seen  that  the  model  cannot  support  Sharpshooter  or  Expert  live-fire 
predictions  with  a  high  degree  of  probability. 

Even  though  the  model  cannot  confidently  support  either  Sharpshooter  or  Expert  live-fire 
predictions,  it  can  predict  live-fire  qualification  vs  nonqualification.  For  example,  if  a 
marksmanship  trainer  wants  a  soldier  to  have  at  least  a  50%  chance  of  record  fire  qualification, 
then  an  LMTS  Blazer  score  of  27  is  required.  The  chance  of  record  fire  qualification  rises  to  70% 
if  the  soldier’s  Blazer  score  is  33,  and  it  climbs  to  80%  with  a  Blazer  score  of  37.  Notice, 
however,  that  it  tops  out  at  80%.  Even  an  LMTS  score  of  40  fells  short  of  supporting  a  90% 
prediction  of  first-run  qualificatioa  Likewise,  a  perfect  Blazer  score  of  40  produces  no  more 
than  a  40%  probability  of  Sharpshooter  qualification,  and  no  more  than  a  10%  probability  of 
Expert  qualificatioa 

Table  2. 

LMTS  Blazer-Based  Predicted  Chances  of  First-Run  Qualification  at  Marksman  (223  hits). 
Sharpshooter  ( >30  hits),  and  Expert  (2  36  hits)  Levels  on  a  Standard  Pop-up  Target 
Qualification  Course^  _ _ _ 


Chances  of  a  Live-Fire  Score  of . . . 


LMTS 

Predicted  Mean 

>23 

>30 

>36 

Score 

Record  Fire  Score 

Marksman 

Sharpshooter 

Expert 

12 

16 

10 

— 

— 

17 

18 

20 

— 

/  — 

21 

20 

30 

— 

24 

22 

40 

“ 

27 

23 

50 

10 

— 

30 

24 

60 

— 

~ 

32 

25 

— 

20 

— 

33 

26 

70 

— 

— 

36 

27 

— 

30 

— 

37 

28 

80 

— 

— 

40 

29 

— 

40 

10 

Discussion  and  Conclusions 

Overall,  these  findings  show  that  (a)  positive  linear  relations  exist  between  simulated  rifle 
marksmanship  performance  on  LMTS  mini-RETS  and  live-fire  rifle  marksmanship  performance 
on  automated  (e.g.,  RETS)  standard  Army  pop-up  target  ranges,  (b)  these  relations  exist  for  both 
dry-fire-  and  Blazer-Based  simulation,  and  (c)  these  relations  are  both  reliable  and  of  sufficient 
magnitude  to  support  implementation  of  an  LMTS -based  tool  for  predicting  first-run,  live-fire 
qualification  rates. 

Moreover,  the  results  were  consistent  with  earlier  findings  (e.g.,  Smith  &  Hagman,  2000; 
2001)  that  established  the  ability  of  LMTS  performance  to  predict  ALT-C  live-fire  qualification 
scores/probabilities.  Thus,  the  scores  from  both  ALT-C  and  standard  pop-up  target  qualification 
courses  have  been  successfully  predicted  through  use  of  LMTS-based  simulation.  To  date,  ALT- 
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C  predictions  have  only  been  validated  for  LMTS  dry-fire,  whereas  pop-up  target  qualification 
course  predictions  have  now  been  validated  for  both  dry-  and  Blazer-Based  fire. 

In  both  current  experiments,  obtained  predictive  relations  were  of  sufficient  strength  to  be 
practically  useful  in  addition  to  statistically  significant,  permitting  the  prediction  of  first-run 
qualification  with  up  to  90%  confidence  using  dry-fire-based  scores  (Table  1)  and  with  up  to 
80%  confidence  using  Blazer-Based  scores  (Table  2).  Thus,  with  either  LMTS  dry-  or  Blazer- 
Based  fire,  it  is  possible  for  marksmanship  trainers  to  set  the  minimum  acceptable  probability  of 
first-run,  live-fire  qualification  (either  for  individual  soldiers  or  for  the  unit  as  a  whole)  and  then 
conduct  LMTS-based  training/evaluation  to  meet  or  exceed  that  standard. 

Simulation  Methods:  Dry-Fire  vs.  Blazer-Based  Fire 

Firing  LMTS  under  a  dry-fire  mode  is  less  expensive,  less  complex,  and  less  prone  to 
malfunction  than  firing  LMTS  under  a  Blazer-Based  mode,  primarily  because  the  former 
requires  less  equipment.  Under  the  dry-fire  mode,  soldiers  use  their  own  (intact)  standard-issue 
weapons  equipped  with  a  laser-emitting  barrel  insert/attachment.  Under  the  Blazer-Based  mode, 
however,  a  weapon’s  upper  receiver  assembly  must  be  replaced  by  the  LMTS  Blazer  assembly. 
While  the  replacement  process  is  not  difficult,  and  while  the  Blazer  assembly  is  realistic  in 
appearance,  weight,  balance,  and  (simulated)  function,  firing  with  it  involves  additional  cost 
(e.g.,  for  the  assemblies  themselves  and  the  nontoxic  blanks  they  fire).  Moreover,  Blazer 
assemblies  require  additional  storage  space,  require  maintenance  during  operation,  and  were 
observed  to  be  somewhat  susceptible  to  malfunction. 

Thus,  although  the  Blazer-Based  mode  of  fire  has  the  potential  of  delivering  enhanced 
realism,  including  features  such  as  semi-automatic  ammunition  cycling,  spent  cartridge  .ejection, 
recoil,  sound  effects,  and  smoke,  these  added  features  are  achieved  at  additional  cost  without 
additional  predictive  benefits.  That  is,  live-fire  predictions  derived  from  Blazer-Based  firing 
were  no  more  accurate  than  predictions  associated  with  dry-fire.  Thus,  the  dry-fire  method  of 
LMTS-based  simulation  seems  preferable  at  this  time.  The  dry-fire  version  of  LMTS  mini-RETS 
works  now  and  provides  a  viable  marksmanship  training  and  evaluation  tool  that  is  ready  for 
immediate  implementation. 

LMTS  Implementation 

Based  on  the  results  of  this  and  other  supporting  investigations  (e.g.,  Smith  &  Hagman, 
2000;  2001),  it  is  clear  that  LMTS  can  be  used  to  predict  first-run,  live-fire  qualification  scores 
with  a  reasonably  high  degree  of  success,  and  that  this  predictive  capability  can  be  applied  either 
to  the  standard  Army  pop-up  target  course  of  fire,  as  demonstrated  here,  or  to  ALT-C,  as 
demonstrated  earlier  (Smith  &  Hagman,  2000).  When  this  predictive  capability  is  coupled  with 
the  previously  developed  and  validated  (Hagman,  2000)  LMTS-based  rifle  marksmanship 
program  of  instruction  (POI)  (Headquarters,  Department  of  the  Army,  2001,  August),  the  result 
is  a  comprehensive  package  o f  LMTS-based  technology  that  can  be  used  as  an  effective 
diagnostic  tool  for  (a)  identifying  soldiers  most  in  need  of  remediation/sustainment  training,  (b) 
signaling  when  enough  such  training  has  been  provided,  and  (c)  providing  simulation-based, . 
live-fire  performance  standards  for  enabling  the  substitution  of  LMTS-based  qualification  firing 
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for  live-fire  qualification  firing  when  outdoor  range  facilities  are  not  readily  available.  In  doing 
so,  these  tools  can  form  the  basis  for  competency-based  instructional  delivery  within  the  USAR's 
newly  developed  rifle  marksmanship  POI. 

Figure  5  shows,  in  flowchart  format,  how  this  competency-based  delivery  approach  would 
work.  For  example,  soldiers  would  first  be  pretested  by  dry  firing  with  LMTS  on  simulated  mini- 
RETS  (or  ALT-C).  Based  on  their  pretest  scores,  soldiers  would  receive  either  a  "Go"  or  "NoGo" 
depending  upon  which  L MTS-based  cut-off  score  were  set  beforehand  by  the  unit  commander. 
Let’s  say,  for  instance,  that  the  LMTS  pretest  cut-off  score  was  set  at  28,  the  score  associated 
with  an  80%  probability  of  live-fire  qualification  at  the  Marksman  level  (Table  1).  Soldiers  firing 
at  or  above  this  cut  off  would  receive  a  Go  and  be  considered  simulation  qualified.  Soldiers 
firing  below  the  cut  off  would  be  identified  as  needing  remediation  (to  be  delivered  via  the 
current  LMTS-based  POI).  Remediation  is  provided  only  for  those  in  need  of  it,  thereby,  making 
the  most  of  valuable  training  time  while  saving  range  time  and  ammunition  in  the  process.  Those 
completing  remediation  would  then  be  posttested  on  LMTS.  Those  receiving  a  Go  on  the  posttest 
would  be  considered  simulation  qualified,  whereas  those  receiving  a  NoGo  would  undergo 
further  remediation  until  they  were  able  to  meet  the  posttest  cut-off  score  and  its  associated  live- 
fire  expectancy  standard  of  80%  probability  of  live-fire  qualification. 


Figure  5.  Flowchart  of  delivery  strategy 

The  LMTS  prediction  tool  (Table  1  for  dry  fire;  Table  2  for  Blazer)  provides  an  empirically 
derived  set  of  marksmanship  performance  probabilities  for  use  in  determining  live-fire 
qualification  standards.  Such  standards,  in  the  form  of  cut-off  scores,  would  be  required  to 
support  a  decision  to  use  LMTS  to  validate  live-fire  marksmanship  proficiency.  Unit  standards 
could  be  set  at  an  acceptable  probability  level,  using  the  values  in  Column  3  of  either  Table  1  or 
2.  Standards  could  be  changed  over  time  to  reflect  unit  proficiency,  or  different  standards  might 
be  applied  to  different  unit  subgroups. 

This  competency-based  delivery  strategy  for  LMTS-based  marksmanship  training,  with 
empirically  derived  live-fire  performance  standards  serving  as  its  basis,  should  enable  the  USAR 
to  take  a  substantial  step  forward  in  its  ongoing  commitment  to  meeting  the  Total  Army 
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readiness  challenge  through  more  productive  home-station  small  arms  marksmanship  training 
and  evaluation.  Current  LMTS  technology  has  the  potential  of  optimizing  the  payoff  from  each 
soldier’s  live-fire  range  experience,  while  conserving  ammunition  and  ensuring  that  scarce 
training  resources  are  devoted  to  those  most  in  need.  With  their  marksmanship  skills  at  the  ready. 
Reserve  Component  soldiers  will  be  better  prepared  to  mobilize  and  deploy  on  short  notice  when 
called  upon  to  support  worldwide  contingency  operations. 
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